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DETAILED ACTION 

Remarks 

1 . Receipt of Applicant's Amendment, filed on 08/1 3/2008, is acknowledged. The 
amendment includes the cancellation of claims 1-6 and the amending of claims 7 and 
11. 

Priority 

2. Receipt is acknowledged of papers submitted under 35 U.S.C. 1 1 9(a)-(d), which 
papers have been placed of record in the file. 

Claim Objections 

3. Claim 7 is objected to because of the following informalities: Applicant is 
reminded that all claims must only have one period. Claim 7 has two periods. 
Appropriate correction is required. 

Claims 8-10 are objected to for incorporating the deficiencies of independent 
claim 7. 

Claim Rejections - 35 USC § 102 

4. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 1 02 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(a) the invention was known or used by others in this country, or patented or described in a printed 
publication in this or a foreign country, before the invention thereof by the applicant for a patent. 

5. Claims 7-13 are rejected under 35 U.S.C. 102(a) as being anticipated by Sinclair 
et al. (U.S. PGPUB 2002/0052551). 

6. Regarding claim 7, Sinclair teaches a method comprising: 

A) determining deviations from the contextual information of a stored comparison 
image and/or of a standard image created by evaluating a plurality of comparison 
images of a similar pathology (Paragraphs 19, 166-167, 231-232, 234, and 252-267); 
and/or 

B) carrying out a similarity analysis from the contextual information of a stored 
comparison image, and/or of a standard image created by evaluating a plurality of 
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comparison images of a similar pathology (Paragraphs 19, 72, 166-167, 231-232, 234, 
and 252-267); and 

C) creating new images that are stored for purposes of comparison at a later time 
(Paragraphs 44 and 72); and 

D) wherein the contextual information is drawn from the settings of the fundus camera, 
manual annotations associated with the recorded fundus images, patient-specific 
information, and image contents (Paragraphs 1 1 9, 1 31 , 1 34, 1 40, 228, and 238). 

The examiner notes that Sinclair teaches "determining deviations from the 
contextual information of a stored comparison image and/or of a standard image 
created by evaluating a plurality of comparison images of a similar pathology" as 
"Another significant element of this invention is one or more retinal grading algorithms 
that automatically evaluate the digital retinal images obtained by the screening 
subsystems for particular retinopathies. Generally, the RGAs operate in a lesion-based 
fashion, first identifying ophthalmologically significant retinal lesions or features by use 
of image processing methods, and second evaluating and grading the retinopathy in 
view of the identified lesions by use of artificial intelligence/cognitive decision 
capabilities" (Paragraph 19), "The RGAs are based on detecting and identifying 
"lesions" in fundus images. Therefore, each image (field of view) is evaluated to detect 
the number and type of lesions, and the cumulative lesion information for all acquired 
images is processed to arrive at a final retinopathy grade level for each eye. This 
processing may be by an expert system, perhaps rule-based, that simulates the 
considerations of an ophthalmologist when presented with similar cumulative lesion 
information" (Paragraph 231), and "In somewhat more detail, the following lists DR 
lesions that are preferably detected and identified by all RGA algorithms. Sophisticated 
RGA algorithms for DR detect additionally the advantageous lesions. DR Lesions and 
Characteristics Preferably Identified... Fovea (or approximate foveal location)... Epi- 
retinal (or epi-papillary) neovascularization-size and distance to optic nerve head or 
fovea" (Paragraphs 252-265). The examiner further notes that Sinclair teaches 
"carrying out a similarity analysis from the contextual information of a stored 
comparison image, and/or of a standard image created by evaluating a plurality of 
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comparison images of a similar pathology" as "Also, in preferred embodiments 
where prior retinal images are available and may be compared to a current retinal 
image, the time progression or regression of lesions may be identified. Then, detailed 
lesion information and lesion history may be taken into account in adjusting retinal 
image grading. For example, if a current image received a grade of level 2, but it 
contained lesions in critical anatomic regions as near the optic nerve head, or the fovea, 
or so forth, or contained rapidly growing or multiplying lesions, it may be promoted to 
grade level 3. Conversely, if a current image received a grade of level 3, but it contained 
regressing lesions in locations posing no threat of imminent visual impairment, it may be 
demoted to grade level 2 (or 2+) (Generally, grade level 3 signifies specialist 
consultation is recommended, while grade level 2 signifies that routine follow-up 
screening is recommended.)" (Paragraph 72), "The RGAs are based on detecting and 
identifying "lesions" in fundus images. Therefore, each image (field of view) is evaluated 
to detect the number and type of lesions, and the cumulative lesion information for all 
acquired images is processed to arrive at a final retinopathy grade level for each eye. 
This processing may be by an expert system, perhaps rule-based, that simulates the 
considerations of an ophthalmologist when presented with similar cumulative lesion 
information" (Paragraph 231), and "In somewhat more detail, the following lists DR 
lesions that are preferably detected and identified by all RGA algorithms. Sophisticated 
RGA algorithms for DR detect additionally the advantageous lesions. DR Lesions and 
Characteristics Preferably Identified... Fovea (or approximate foveal location)... Epi- 
retinal (or epi-papillary) neovascularization-size and distance to optic nerve head or 
fovea" (Paragraphs 252-265). The examiner further notes that Sinclair teaches 
"creating new images that are stored for purposes of comparison at a later time" 
as "identify, and characterize in the prior retinal images lesions from the pre-determined 
set lesions type, comparing the lesions detected in the image taken at the selected time 
with the lesions detected in the prior image to detect changes in the lesions, and 
performing a decision process that assigns a grade to the retinal image taken at the 
selected time in dependence on the identities and characteristics of the lesions detected 
in that image, and in dependence on the changes in the lesions detected in the 
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comparing step" (Paragraph 44) and "Also, in preferred embodiments where prior retinal 
images are available and may be compared to a current retinal image, the time 
progression or regression of lesions may be identified. Then, detailed lesion information 
and lesion history may be taken into account in adjusting retinal image grading. For 
example, if a current image received a grade of level 2, but it contained lesions in critical 
anatomic regions as near the optic nerve head, or the fovea, or so forth, or contained 
rapidly growing or multiplying lesions, it may be promoted to grade level 3. Conversely, 
if a current image received a grade of level 3, but it contained regressing lesions in 
locations posing no threat of imminent visual impairment, it may be demoted to grade 
level 2 (or 2+) (Generally, grade level 3 signifies specialist consultation is 
recommended, while grade level 2 signifies that routine follow-up screening is 
recommended.)" (Paragraph 72). The examiner further notes that Sinclair teaches 
" wherein the contextual information is drawn from the settings of the fundus 
camera, manual annotations associated with the recorded fundus images, 
patient-specific information, and image contents " as "The central database ("CDB") 
is an on-line (or otherwise efficiently accessible) storage repository of the data 
generated in an OSS system. The CDB stores patient oriented data such as original 
image data from patient screening examinations, results of RGA screening including 
images annotated or marked-up with lesion identification, associated patient 
identification, demographics, and screening/examination history, results of manual 
ophthalmologist grading process including any annotated images, referrals and reports. 
This database also stores system oriented data such as statistical data gathered from 
analysis of the patient data, results of the image quality assessment process, the Yules' 
to be used by the WFM for handling images, reports, and messages" (Paragraph 119), 
"Screening session identification (screening site identification, date, time, confirmation 
of patient data, race (affects image processing parameters), sex, photographer, camera 
utilized and type of images acquired)", (Paragraph 131), "Screening session 
identification (screening site identification, date, time, confirmation of patient data, race 
(affects image processing parameters), sex, photographer, camera utilized and type of 
images acquired)" (Paragraph 134), "Equipment available (cameras, other resources)" 
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(Paragraph 140), "RGA results from complete evaluation of all fundus images are stored 
in the Central Database. Preferably, RGA results include evaluated image annotated or 
marked-up with indicia to identify, e.g., the position or the identity of detected lesions. In 
cases of doubt, the annotation may include indications of "definitely a lesion," or 
"possibly a lesion." Annotations can include highlighting, coloring, outlining, pointing 
with arrows or the equivalent, and other methods known in the art (such as text 
superimposed on the image). Color coding of lesion characteristics may be used to 
simplify interpreting the annotations. The annotated images are saved (using an 
appropriate naming convention) along with the original images in the CDB" (Paragraph 
228), and "Finally, RGAs have been discovered to be dependent on camera properties, 
digital image pixel density, depth and the magnification, and so forth. This dependence 
is preferably factored into RGA processing, for example, by inverse transforming known 
effects from the image" (Paragraph 238). 

Regarding claim 8, Sinclair further teaches a method comprising: 
A) wherein the evaluation is carried out by averaging extracted features (Paragraphs 
305-344). 

The examiner notes that Sinclair teaches "wherein the evaluation is carried 
out by averaging extracted features" as "The following lists exemplary and non- 
limiting statistical information which may be obtained and accumulated in an OSS 
implementation. The following statistical parameters may be accumulated to aid in 
quality control and oversight of an OSS. Exemplary Quality Control Statistics... Percent 
of eyes with more advanced lesions noted in each field (1 , 2, 3, 4 or 5) without 
equivalent lesions noted in other fields. Percent of patients who complied with 
recommendation for screening and follow-up screening; time interval between receipt of 
recommendation/referral by patient and actual follow-up screening. Sensitivity & 
specificity of the grading algorithm as compared with the gold standard of grading 
performed by the retinal special ist— for each eye (variance with age, pupil size, necessity 
for dilation, presence of cataract)" (Paragraphs 305-344). The examiner further notes 
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that the various statistical analysis of the fundus properties of patients is analogous to 
an averaging function. 

Regarding claim 9, Sinclair further teaches a method comprising: 
A) wherein deviations are determined and/or the similarity analysis is carried out on the 
basis of a gray-value analysis and/or an analysis of color histograms and/or a structure 
analysis (Paragraphs 19, 166-167, 231-232, 234, and 252-267). 

The examiner notes that Sinclair teaches "wherein deviations are determined 
and/or the similarity analysis is carried out on the basis of a gray-value analysis 
and/or an analysis of color histograms and/or a structure analysis" as "Another 
significant element of this invention is one or more retinal grading algorithms that 
automatically evaluate the digital retinal images obtained by the screening subsystems 
for particular retinopathies. Generally, the RGAs operate in a lesion-based fashion, first 
identifying ophthalmologically significant retinal lesions or features by use of image 
processing methods, and second evaluating and grading the retinopathy in view of the 
identified lesions by use of artificial intelligence/cognitive decision capabilities" 
(Paragraph 19), "The RGAs are based on detecting and identifying "lesions" in fundus 
images. Therefore, each image (field of view) is evaluated to detect the number and 
type of lesions, and the cumulative lesion information for all acquired images is 
processed to arrive at a final retinopathy grade level for each eye. This processing may 
be by an expert system, perhaps rule-based, that simulates the considerations of an 
ophthalmologist when presented with similar cumulative lesion information" (Paragraph 
231), and "In somewhat more detail, the following lists DR lesions that are preferably 
detected and identified by all RGA algorithms. Sophisticated RGA algorithms for DR 
detect additionally the advantageous lesions. DR Lesions and Characteristics 
Preferably Identified... Fovea (or approximate foveal location)... Epi-retinal (orepi- 
papillary) neovascularization-size and distance to optic nerve head or fovea" 
(Paragraphs 252-265). The examiner further notes that paragraph 1 1 of the 
specification of the instant application defines a structural analysis of as "classification 
and quantification of structures at the ocular fundus (e.g., papilla, fovea)". 
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Regarding claim 10, Sinclair further teaches a method comprising: 
A) wherein an extraction of vascular tree parameters is carried out (Paragraphs 268- 
272). 

The examiner notes that Sinclair teaches "wherein deviations are determined 
and/or the similarity analysis is carried out on the basis of a gray-value analysis 
and/or an analysis of color histograms and/or a structure analysis" as "Second, 
diameter and tortuosity measurements for major vessel abnormalities including: Major 
artery tortuosity-deviations of 1 .sup.st, 2.sup.nd, and 3.sup.rd order arteries from a 
straight line (point-to-point); also requires determination of whether the deviations are 
caused by branchings or by deviations between branchings; in other words, if a vessel 
branches unequally (daughter vessels are unequal in caliber), this causes a deviation of 
the large parent vessel into the larger of the two daughter vessels, Major vein tortuosity- 
-deviations of 1 .sup.st, 2.sup.nd, and 3.sup.rd order veins from a straight line (point-to- 
point) and whether deviations are caused by branchings or by deviations in between 
branchings, Major artery diameter (and variation in diameter) versus distance along 
vessel starting at the optic nerve head-for 1 .sup.st and 2.sup.nd order vessels; second 
order vessels are defined as either two daughter vessels after an equal branching 
(branching in which both daughter vessels are of same caliber) or the smaller caliber 
vessel of the daughter vessels in an unequal branching, Major vein diameter (and 
variation in diameter) versus distance along vessel-for 1. sup.st and 2.sup.nd order 
vessels." (Paragraphs 268-272). 

Regarding claim 1 1 , Sinclair teaches a system comprising: 

A) a fundus camera for recording the ocular fundus (Paragraphs 88-89, and 184); 

B) an image storage for storing recorded fundus images (Paragraphs 24, 119-121, 1 66, 
and 228); and 

C) means for evaluating for analyzing the images according to the same or similar 
contextual information, the recorded fundus images of a similar pathology further 
comprising means for gray-value analysis and/or means for preparing color histograms 
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and/or means for structure analysis (Paragraphs 19, 166-167, 231-232, 234, and 252- 
267); 

D) a comparison unit connected to the image storage (Paragraphs 44 and 72); and 

E) wherein the contextual information is drawn from the settings of the fundus camera, 
manual annotations associated with the recorded fundus images, patient-specific 
information, and image contents (Paragraphs 1 1 9, 1 31 , 1 34, 1 40, 228, and 238); and 

F) wherein the comparison unit compares images recording in the image storage and 
creates new images of a similar pathology (Paragraphs 44 and 72). 

The examiner notes that Sinclair teaches "a fundus camera for recording the 
ocular fundus" as "For image capture and acquisition, preferably a non-mydriatic 
retinal camera is used to acquire retinal images in order to avoid the patient 
inconvenience of pupil dilation (mydriasis)" (Paragraph 88), "The present invention may 
use a wide range of non-mydriatic cameras, including commercially-available cameras 
from, e.g., Canon, Nikon, and so forth, and also including specially designed and built 
cameras. From whatever source, preferred cameras have should have optics capable of 
acquiring up to 45.degree. retinal fields through pupils down to 2.0 mm in diameter with 
adequate image contrast and resolution" (Paragraph 89), and "retinal (fundus) camera 
with CCD sensors" (Paragraph 184). The examiner further notes that Sinclair teaches 
"an image storage for storing recorded fundus images" as "The central database 
("CDB") is an on-line (or otherwise efficiently accessible) storage repository of the data 
generated in an OSS system. The CDB stores patient oriented data such as original 
image data from patient screening examinations, results of RGA screening including 
images annotated or marked-up with lesion identification, associated patient 
identification, demographics, and screening/examination history, results of manual 
ophthalmologist grading process including any annotated images, referrals and reports" 
(Paragraph 119) and "The CDB has several uses in an OSS, and its centralized image 
(also possible with distributed database architectures) provides several advantages. Its 
principal use is to provide physicians, specialists, ophthalmologists, and other users 
with access to current images as well as the results of any prior studies, regardless of 
where acquired. This historical record permits an objective and quantitative evaluation, 
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either by automatic algorithmic processes or by manual physician examination, of the 
status and progression of the ocular disease in individual patients" (Paragraph 121). 
The examiner further notes that Sinclair teaches "means for evaluating for analyzing 
the images according to the same or similar contextual information, the recorded 
fundus images of a similar pathology further comprising means for gray-value 
analysis and/or means for preparing color histograms and/or means for structure 
analysis" as "Another significant element of this invention is one or more retinal grading 
algorithms that automatically evaluate the digital retinal images obtained by the 
screening subsystems for particular retinopathies. Generally, the RGAs operate in a 
lesion-based fashion, first identifying ophthalmologically significant retinal lesions or 
features by use of image processing methods, and second evaluating and grading the 
retinopathy in view of the identified lesions by use of artificial intelligence/cognitive 
decision capabilities" (Paragraph 19), "The RGAs are based on detecting and identifying 
"lesions" in fundus images. Therefore, each image (field of view) is evaluated to detect 
the number and type of lesions, and the cumulative lesion information for all acquired 
images is processed to arrive at a final retinopathy grade level for each eye. This 
processing may be by an expert system, perhaps rule-based, that simulates the 
considerations of an ophthalmologist when presented with similar cumulative lesion 
information" (Paragraph 231), and "In somewhat more detail, the following lists DR 
lesions that are preferably detected and identified by all RGA algorithms. Sophisticated 
RGA algorithms for DR detect additionally the advantageous lesions. DR Lesions and 
Characteristics Preferably Identified... Fovea (or approximate foveal location)... Epi- 
retinal (or epi-papillary) neovascularization-size and distance to optic nerve head or 
fovea" (Paragraphs 252-265). The examiner further notes that paragraph 1 1 of the 
specification of the instant application defines a structural analysis of as "classification 
and quantification of structures at the ocular fundus (e.g., papilla, fovea)". The 
examiner further notes that Sinclair teaches "a comparison unit connected to the 
image storage" as "identify, and characterize in the prior retinal images lesions from 
the pre-determined set lesions type, comparing the lesions detected in the image taken 
at the selected time with the lesions detected in the prior image to detect changes in the 
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lesions, and performing a decision process that assigns a grade to the retinal image 
taken at the selected time in dependence on the identities and characteristics of the 
lesions detected in that image, and in dependence on the changes in the lesions 
detected in the comparing step" (Paragraph 44) and "Also, in preferred embodiments 
where prior retinal images are available and may be compared to a current retinal 
image, the time progression or regression of lesions may be identified. Then, detailed 
lesion information and lesion history may be taken into account in adjusting retinal 
image grading. For example, if a current image received a grade of level 2, but it 
contained lesions in critical anatomic regions as near the optic nerve head, or the fovea, 
or so forth, or contained rapidly growing or multiplying lesions, it may be promoted to 
grade level 3. Conversely, if a current image received a grade of level 3, but it contained 
regressing lesions in locations posing no threat of imminent visual impairment, it may be 
demoted to grade level 2 (or 2+) (Generally, grade level 3 signifies specialist 
consultation is recommended, while grade level 2 signifies that routine follow-up 
screening is recommended.)" (Paragraph 72). The examiner further notes that Sinclair 
teaches " wherein the contextual information is drawn from the settings of the 
fundus camera, manual annotations associated with the recorded fundus images, 
patient-specific information, and image contents " as "The central database ("CDB") 
is an on-line (or otherwise efficiently accessible) storage repository of the data 
generated in an OSS system. The CDB stores patient oriented data such as original 
image data from patient screening examinations, results of RGA screening including 
images annotated or marked-up with lesion identification, associated patient 
identification, demographics, and screening/examination history, results of manual 
ophthalmologist grading process including any annotated images, referrals and reports. 
This database also stores system oriented data such as statistical data gathered from 
analysis of the patient data, results of the image quality assessment process, the Yules' 
to be used by the WFM for handling images, reports, and messages" (Paragraph 119), 
"Screening session identification (screening site identification, date, time, confirmation 
of patient data, race (affects image processing parameters), sex, photographer, camera 
utilized and type of images acquired)", (Paragraph 131), "Screening session 
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identification (screening site identification, date, time, confirmation of patient data, race 
(affects image processing parameters), sex, photographer, camera utilized and type of 
images acquired)" (Paragraph 134), "Equipment available (cameras, other resources)" 
(Paragraph 140), "RGA results from complete evaluation of all fundus images are stored 
in the Central Database. Preferably, RGA results include evaluated image annotated or 
marked-up with indicia to identify, e.g., the position or the identity of detected lesions. In 
cases of doubt, the annotation may include indications of "definitely a lesion," or 
"possibly a lesion." Annotations can include highlighting, coloring, outlining, pointing 
with arrows or the equivalent, and other methods known in the art (such as text 
superimposed on the image). Color coding of lesion characteristics may be used to 
simplify interpreting the annotations. The annotated images are saved (using an 
appropriate naming convention) along with the original images in the CDB" (Paragraph 
228), and "Finally, RGAs have been discovered to be dependent on camera properties, 
digital image pixel density, depth and the magnification, and so forth. This dependence 
is preferably factored into RGA processing, for example, by inverse transforming known 
effects from the image" (Paragraph 238). The examiner further notes that Sinclair 
teaches "wherein the comparison unit compares images recording in the image 
storage and creates new images of a similar pathology" as "identify, and 
characterize in the prior retinal images lesions from the pre-determined set lesions type, 
comparing the lesions detected in the image taken at the selected time with the lesions 
detected in the prior image to detect changes in the lesions, and performing a decision 
process that assigns a grade to the retinal image taken at the selected time in 
dependence on the identities and characteristics of the lesions detected in that image, 
and in dependence on the changes in the lesions detected in the comparing step" 
(Paragraph 44) and "Also, in preferred embodiments where prior retinal images are 
available and may be compared to a current retinal image, the time progression or 
regression of lesions may be identified. Then, detailed lesion information and lesion 
history may be taken into account in adjusting retinal image grading. For example, if a 
current image received a grade of level 2, but it contained lesions in critical anatomic 
regions as near the optic nerve head, or the fovea, or so forth, or contained rapidly 
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growing or multiplying lesions, it may be promoted to grade level 3. Conversely, if a 
current image received a grade of level 3, but it contained regressing lesions in 
locations posing no threat of imminent visual impairment, it may be demoted to grade 
level 2 (or 2+) (Generally, grade level 3 signifies specialist consultation is 
recommended, while grade level 2 signifies that routine follow-up screening is 
recommended.)" (Paragraph 72). 

Regarding claim 12, Sinclair further teaches a system comprising: 
A) a means for determining deviations from a stored comparison image and/or from a 
standard image created by evaluating a plurality of comparison images, and/or a means 
for carrying out a similarity analysis by a stored comparison image and/or by a standard 
image created by evaluating a plurality of comparison images (Paragraphs 19, 166-167, 
231-232, 234, and 252-267). 

The examiner notes that Sinclair teaches "a means for determining 
deviations from a stored comparison image and/or from a standard image created 
by evaluating a plurality of comparison images, and/or a means for carrying out a 
similarity analysis by a stored comparison image and/or by a standard image 
created by evaluating a plurality of comparison images" as "Another significant 
element of this invention is one or more retinal grading algorithms that automatically 
evaluate the digital retinal images obtained by the screening subsystems for particular 
retinopathies. Generally, the RGAs operate in a lesion-based fashion, first identifying 
ophthalmologically significant retinal lesions or features by use of image processing 
methods, and second evaluating and grading the retinopathy in view of the identified 
lesions by use of artificial intelligence/cognitive decision capabilities" (Paragraph 19), 
"The RGAs are based on detecting and identifying "lesions" in fundus images. 
Therefore, each image (field of view) is evaluated to detect the number and type of 
lesions, and the cumulative lesion information for all acquired images is processed to 
arrive at a final retinopathy grade level for each eye. This processing may be by an 
expert system, perhaps rule-based, that simulates the considerations of an 
ophthalmologist when presented with similar cumulative lesion information" (Paragraph 
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231), and "In somewhat more detail, the following lists DR lesions that are preferably 
detected and identified by all RGA algorithms. Sophisticated RGA algorithms for DR 
detect additionally the advantageous lesions. DR Lesions and Characteristics 
Preferably Identified... Fovea (or approximate foveal location)... Epi-retinal (orepi- 
papillary) neovascularization-size and distance to optic nerve head or fovea" 
(Paragraphs 252-265). 

Regarding claim 13, Sinclair further teaches a system comprising: 
A) wherein means are provided for determining deviations from a stored comparison 
image and/or from a standard image created by evaluating a plurality of comparison 
images, and/or means are provided for similarity analysis by a stored comparison image 
and/or a standard image created by evaluating a plurality of comparison images 
(Paragraphs 19, 166-167, 231-232, 234, and 252-267). 

The examiner notes that Sinclair teaches "wherein means are provided for 
determining deviations from a stored comparison image and/or from a standard 
image created by evaluating a plurality of comparison images, and/or means are 
provided for similarity analysis by a stored comparison image and/or a standard 
image created by evaluating a plurality of comparison images" as "Another 
significant element of this invention is one or more retinal grading algorithms that 
automatically evaluate the digital retinal images obtained by the screening subsystems 
for particular retinopathies. Generally, the RGAs operate in a lesion-based fashion, first 
identifying ophthalmologically significant retinal lesions or features by use of image 
processing methods, and second evaluating and grading the retinopathy in view of the 
identified lesions by use of artificial intelligence/cognitive decision capabilities" 
(Paragraph 19), "The RGAs are based on detecting and identifying "lesions" in fundus 
images. Therefore, each image (field of view) is evaluated to detect the number and 
type of lesions, and the cumulative lesion information for all acquired images is 
processed to arrive at a final retinopathy grade level for each eye. This processing may 
be by an expert system, perhaps rule-based, that simulates the considerations of an 
ophthalmologist when presented with similar cumulative lesion information" (Paragraph 
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231), and "In somewhat more detail, the following lists DR lesions that are preferably 
detected and identified by all RGA algorithms. Sophisticated RGA algorithms for DR 
detect additionally the advantageous lesions. DR Lesions and Characteristics 
Preferably Identified... Fovea (or approximate foveal location)... Epi-retinal (orepi- 
papillary) neovascularization-size and distance to optic nerve head or fovea" 
(Paragraphs 252-265). 

Response to Arguments 

7. Applicant's arguments filed 08/13/2008 have been fully considered but they are 
not persuasive. 

Applicants argue on page 4 that "Sinclair does not disclose analyzing images 
according to contextual information. For example, Sinclair does not discuss 
analyzing "settings of the fundus camera", or "manual annotations associated 
with the recorded fundus images". Rather, Sinclair teaches finding. ..diagnosis". 

However, the examiner wishes to refer to Paragraphs 119, 131, 134, 140, 228, and 238 
of Sinclair which state "The central database ("CDB") is an on-line (or otherwise 
efficiently accessible) storage repository of the data generated in an OSS system. The 
CDB stores patient oriented data such as original image data from patient screening 
examinations, results of RGA screening including images annotated or marked-up with 
lesion identification, associated patient identification, demographics, and 
screening/examination history, results of manual ophthalmologist grading process 
including any annotated images, referrals and reports. This database also stores 
system oriented data such as statistical data gathered from analysis of the patient data, 
results of the image quality assessment process, the Yules' to be used by the WFM for 
handling images, reports, and messages" (Paragraph 119), "Screening session 
identification (screening site identification, date, time, confirmation of patient data, race 
(affects image processing parameters), sex, photographer, camera utilized and type of 
images acquired)", (Paragraph 131), "Screening session identification (screening site 
identification, date, time, confirmation of patient data, race (affects image processing 
parameters), sex, photographer, camera utilized and type of images acquired)" 
(Paragraph 134), "Equipment available (cameras, other resources)" (Paragraph 140), 
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"RGA results from complete evaluation of all fundus images are stored in the Central 
Database. Preferably, RGA results include evaluated image annotated or marked-up 
with indicia to identify, e.g., the position or the identity of detected lesions. In cases of 
doubt, the annotation may include indications of "definitely a lesion," or "possibly a 
lesion." Annotations can include highlighting, coloring, outlining, pointing with arrows or 
the equivalent, and other methods known in the art (such as text superimposed on the 
image). Color coding of lesion characteristics may be used to simplify interpreting the 
annotations. The annotated images are saved (using an appropriate naming 
convention) along with the original images in the CDB" (Paragraph 228), and "Finally, 
RGAs have been discovered to be dependent on camera properties, digital image pixel 
density, depth and the magnification, and so forth. This dependence is preferably 
factored into RGA processing, for example, by inverse transforming known effects from 
the image" (Paragraph 238). The examiner further wishes to state that it is clear that 
Sinclair teaches analyzing via annotations (See "RGA results from complete evaluation 
of all fundus images are stored in the Central Database. Preferably, RGA results include 
evaluated image annotated or marked-up with indicia to identify, e.g., the position or the 
identity of detected lesions. In cases of doubt, the annotation may include indications of 
"definitely a lesion," or "possibly a lesion." Annotations can include highlighting, coloring, 
outlining, pointing with arrows or the equivalent, and other methods known in the art 
(such as text superimposed on the image)") and camera settings (See "RGAs have 
been discovered to be dependent on camera properties, digital image pixel density, 
depth and the magnification, and so forth. This dependence is preferably factored into 
RGA processing, for example, by inverse transforming known effects from the image"). 

Applicants argue on page 5 that "Paragraph [0072] discloses that 
corresponding previous retinal images are selected for comparison of the images 
which are recorded with the fundus camera, analyzed, and classed to analyze the 
progression or regression of injuries. This assumes, however, that the images 
selected for comparison are images of the same eye. The selection is 
accordingly not carried out by a similarity analysis". However, the examiner 
wishes to refer to Paragraphs 19, 72, 166-167, 231-232, 234, and 252-267 of Sinclair 
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which state "Another significant element of this invention is one or more retinal grading 
algorithms that automatically evaluate the digital retinal images obtained by the 
screening subsystems for particular retinopathies. Generally, the RGAs operate in a 
lesion-based fashion, first identifying ophthalmologically significant retinal lesions or 
features by use of image processing methods, and second evaluating and grading the 
retinopathy in view of the identified lesions by use of artificial intelligence/cognitive 
decision capabilities" (Paragraph 19), "Also, in preferred embodiments where prior 
retinal images are available and may be compared to a current retinal image, the time 
progression or regression of lesions may be identified. Then, detailed lesion information 
and lesion history may be taken into account in adjusting retinal image grading. For 
example, if a current image received a grade of level 2, but it contained lesions in critical 
anatomic regions as near the optic nerve head, or the fovea, or so forth, or contained 
rapidly growing or multiplying lesions, it may be promoted to grade level 3. Conversely, 
if a current image received a grade of level 3, but it contained regressing lesions in 
locations posing no threat of imminent visual impairment, it may be demoted to grade 
level 2 (or 2+) (Generally, grade level 3 signifies specialist consultation is 
recommended, while grade level 2 signifies that routine follow-up screening is 
recommended.)" (Paragraph 72), "The RGAs are based on detecting and identifying 
"lesions" in fundus images. Therefore, each image (field of view) is evaluated to detect 
the number and type of lesions, and the cumulative lesion information for all acquired 
images is processed to arrive at a final retinopathy grade level for each eye. This 
processing may be by an expert system, perhaps rule-based, that simulates the 
considerations of an ophthalmologist when presented with similar cumulative lesion 
information" (Paragraph 231), and "In somewhat more detail, the following lists DR 
lesions that are preferably detected and identified by all RGA algorithms. Sophisticated 
RGA algorithms for DR detect additionally the advantageous lesions. DR Lesions and 
Characteristics Preferably Identified... Fovea (or approximate foveal location)... Epi- 
retinal (or epi-papillary) neovascularization-size and distance to optic nerve head or 
fovea" (Paragraphs 252-265). The examiner further wishes to state that it is clear that 
Sinclair teaches a similarity analysis because a new image is compared to other 
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images (See "where prior retinal images are available and may be compared to a 
current retinal image"). 

Conclusion 

8. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 

U.S. Patent 6,757,409 issued to Marshall et al. on 29 June 2004. The subject 
matter disclosed therein is pertinent to that of claims 7-13 (e.g., methods to capture, 
store, analyze, compare, and retrieve fundus images). 

U.S. PGPUB 2004/0156016 issued to Kerr et al. on 12 August 2004. The 
subject matter disclosed therein is pertinent to that of claims 7-13 (e.g., methods to 
capture, store, analyze, compare, and retrieve fundus images). 

U.S. Patent 7,055,955 issued to Kishida et al. on 06 June 2006. The subject 
matter disclosed therein is pertinent to that of claims 7-13 (e.g., methods to capture, 
store, analyze, compare, and retrieve fundus images). 

U.S. Patent 6,755,526 issued to Shibata on 29 June 2004. The subject matter 
disclosed therein is pertinent to that of claims 7-13 (e.g., methods to capture, store, 
analyze, compare, and retrieve fundus images). 

U.S. Patent 6,112,114 issued to Dreheron 29 August 2000. The subject matter 
disclosed therein is pertinent to that of claims 7-13 (e.g., methods to capture, store, 
analyze, compare, and retrieve fundus images). 

U.S. Patent 6,409,342 issued to Ohnyuma et al. on 25 June 2002. The subject 
matter disclosed therein is pertinent to that of claims 7-13 (e.g., methods to capture, 
store, analyze, compare, and retrieve fundus images). 

U.S. Patent 6,928,193 issued to Gersten on 09 August 2005. The subject matter 
disclosed therein is pertinent to that of claims 7-13 (e.g., methods to capture, store, 
analyze, compare, and retrieve fundus images). 

U.S. Patent 5,287,129 issued to Sano et al. on 15 February 1994. The subject 
matter disclosed therein is pertinent to that of claims 7-13 (e.g., methods to capture, 
store, analyze, compare, and retrieve fundus images). 
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U.S. Patent 6,053,865 issued to Sugiyama et al. on 25 April 2000. The subject 
matter disclosed therein is pertinent to that of claims 7-13 (e.g., methods to capture, 
store, analyze, compare, and retrieve fundus images). 

U.S. Patent 5,993,001 issued to Bursell etal. on 13 November 1999. The 
subject matter disclosed therein is pertinent to that of claims 7-13 (e.g., methods to 
capture, store, analyze, compare, and retrieve fundus images). 

U.S. Patent 5,557,9471 issued to Barber et al. on 26 November 1996. The 
subject matter disclosed therein is pertinent to that of claims 7-13 (e.g., methods to 
capture, store, analyze, compare, and retrieve fundus images). 

U.S. Patent 6,766,041 issued to Golden et al. on 20 July 2004. The subject 
matter disclosed therein is pertinent to that of claims 7-13 (e.g., methods to capture, 
store, analyze, compare, and retrieve fundus images). 

U.S. Patent 6,453,057 issued to Marshall et al. on 17 September 2002. The 
subject matter disclosed therein is pertinent to that of claims 7-13 (e.g., methods to 
capture, store, analyze, compare, and retrieve fundus images). 

U.S. Patent 6,714,672 issued to Berestov et al. on 30 March 2004. The subject 
matter disclosed therein is pertinent to that of claims 7-13 (e.g., methods to capture, 
store, analyze, compare, and retrieve fundus images). 

U.S. Patent 6„1 79,421 issued to Pang on 30 January 2001. The subject matter 
disclosed therein is pertinent to that of claims 7-13 (e.g., methods to capture, store, 
analyze, compare, and retrieve fundus images). 

U.S. Patent 6,840,933 issued to Pang et al. on 1 1 January 2005. The subject 
matter disclosed therein is pertinent to that of claims 7-13 (e.g., methods to capture, 
store, analyze, compare, and retrieve fundus images). 

U.S. Patent 7,025,459 issued to Cronsweet et al. on 1 1 April 2006. The subject 
matter disclosed therein is pertinent to that of claims 7-13 (e.g., methods to capture, 
store, analyze, compare, and retrieve fundus images). 

9. Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 
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§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 

Contact Information 
1 0. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Mahesh Dwivedi whose telephone number is (571) 272- 
2731 . The examiner can normally be reached on Monday to Friday 8:20 am - 4:40 pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Tim Vo can be reached (571 ) 272-3642. The fax number for the 
organization where this application or proceeding is assigned is (571) 273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov . Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 

Mahesh Dwivedi 
Patent Examiner 
Art Unit 2168 
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